MIXED_FIT: A SAS Macro to Assess Model Fit and Adequacy for Two-Level Linear Models

As multilevel models (MLMs) are useful in understanding relationships existent in hierarchical data structures, these models have started to be used more frequently in research developed in social and health sciences. In order to draw meaningful conclusions from MLMs, researchers need to make sure that the model fits the data. Model fit, and thus, ultimately model selection can be assessed by examining changes in several fit indices across nested and/or nonnested models [e.g., -2 log likelihood (-2LL), Akaike Information Criterion (AIC), and Schwarz’s Bayesian Information Criterion (BIC)]. In addition, the difference in pseudo-R 2 is often used to examine the practical significance between two nested models. Considering the importance of using all of these measures when determining model selection, researchers who use analyze multilevel models would benefit from being able to easily assess model fit across estimated models. Whereas SAS PROC MIXED produces the -2LL, AIC, and BIC, it does not provide the actual change in these fit indices or the change in pseudo-R 2 between different nested and non-nested models. In order to make this information more attainable, Bardenheier (2009) developed a macro that allowed researchers using PROC MIXED to obtain the test statistic for the difference in -2LL along with the p-value of the Likelihood Ratio Test (LRT). As an extension of Bardenheier’s work, this paper provides a comprehensive SAS macro that incorporates changes in model fit statistics (-2LL, AIC and BIC) as well as change in pseudo-R 2 . By utilizing data from PROC MIXED ODS tables, the macro produces a comprehensive table of changes in model fit measures. Thus, this expanded macro allows SAS users to examine model fit in both nested and non-nested models and both in terms of statistical and practical significance. This paper provides a review of the different methods used to assess model fit in multilevel analysis, the macro programming language, an executed example of the macro, and a copy of the complete macro.